Landsat 7 Science Data Processing: A Systems Overvietv 


Robert J. Schweiss^*®^, Nathaniel E. Daniel^*’\ Deborah K. Derrick^^ 

^*^Goddard Space Flight Center - Code 586, Greenbelt, Maryland 20771 
Computer Sciences Corporation, 7700 Hubble Drive, Lanham-Seabrook, Maryland 20706 


ABSTRACT 

The Landsat 7 Science Data Processing System, developed by NASA for the Landsat 7 Project, provides the science data 
handling infrastructure used at the Earth Resources Observation Systems (EROS) Data Center (EDC) Landsat Data Handling 
Facility (DHF) of the United States Department of Interior, United States Geological Survey (USGS) located m Sioux Falls, 
South Dakota. This paper presents an overview of the Landsat 7 Science Data Processing System and details of the design, 
architecture, concept of operation, and management aspects of systems used in the processing of the Landsat 7 Science Data. 
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1. INTRODUCTION 

The Landsat 7 (L7) Science Data Processing System consists of the following processing systems: Landsat Ground St^ion 
(LGS), Landsat Processing System (LPS). Image Assessment System (IAS), Level 1 Product Generation System (LPG^, 
Level 1 Product Distribution System (LPDS) and the EROS Data Center (EDC) Distributed Active ^chive Center (DAAC) 
Earth Observing System (EOS) Data and Information System (EOSDIS) Core System (ECS). The LGS tracks the satellite 
and acquires the downlink signal for real-time transfer to the LPS. It also receives playback tapes from the EOS Polar 
Ground Network (EPGN). The LPS captures raw wideband data, performs Level Zero processing, generates image, metadata 
and browse files, and stages the data for subsequent transfer to the ECS. The IAS enables the L7 Science Office and Mission 
Management Office to perform radiometric and geometric characterizations and quality assessments of the data. Ihese 
characterizations are used in the generation of the Calibration Parameter File (CPF). The ECS facilitates long-tem data 
archiving; science data search, browse, and order; and Level Zero Reformatted pR) data distribution. The LPGS and 
LPDS work together with the ECS to facilitate on-demand Level 1 product generation, packaging, and distributiori^Figure 1 
illustrates the interconnections of the systems. This paper gives an overview of each of the systems as well as specific details 
about the LPS, IAS, LPDS, and LPGS hardware and software architectures and management aspects of the systems 

development. 
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Figure 1 , Landsat 7 Science Data Processing Block Diagram 
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2. OVERVIEW - LAIWSAT GROUND STATION (LGS) 


The Landsat Ground Station (LGS) is responsible for acquiring the Enhanced Thematic Mapper Plus (ETM+) X-Band signal 
from the Landsat 7 spacecraft via any two of the three possible 150-Mbps X-band downlinks. The LGS equipment receives 
and demodulates each X^band downlink into two 75-Mbps I and Q channels and forwards the data to the LPS. Additionally, 
the LGS can receive and transmit S-Band telemetry and generate two-way Doppler tracking for each pass. The LGS 
interfaces with the Mission Operations Center (MOC) for receipt of spacecraft contact schedules and orbital elements, 
exchange of spacecraft commands and telemetry data, transmission of Doppler tracking data, and forwarding of post pass and 
equipment status reports. 

The LGS antenna system consists of a single 10-meter, S- and X-Band, AZ/EL/Tilt Pedestal antenna, a boresight collimation 
source, and redundant Low-Noise Amplifiers by Datron. Additionally, it has a 200W S-Band transmitter by Microdyne. The 
LGS also utilizes X-Band downconverters, demodulators and bit synchronizers, built by Scientific Atlanta, a PC-based 
Monitor and Control System integrated with FactoryLink COTS software by USDATA Corp., a matrix switch by Matrix 
Switch, and four DIS 1601 recorders by Ampex. “ 

The LGS was designed for minimal operations intervention. Its Monitor and Control System automatically receives 
spacecraft contact and ephemeris information from the MOC for autonomous station configuration and pass acquisition. 
During a spacecraft pass the X-band signal is downlinked, demodulated, bit synchronized, and forwarded to the LPS in real- 
time for raw data capture. The LGS operator, performs pass coordination with the MOC, trouble shoots system anomalies, 
supports systems contingency procedures, and performs routine system maintenance. Additionally, the LGS operator 
schedules and loads tapes from the EPGN for data playback from the Ampex recorders to the LGS for subsequent capture 
and processing by the LPS. 


3. OVERVIEW - LANDSAT PROCESSING SYSTEM (LPS) 

The Landsat Processing System (LPS) is responsible for raw ETM+ science data capture, Level OR processing, imagery 
generation, and the subsequent transfer of all imagery to the ECS for long-term archiving and distribution. The LPS was 
designed to capture and process the equivalent of 275 scenes (i.e., 140 GB) of science data per day with no more than a 16- 
hour latency from data acquisition to LOR processing completion. 

The major functions of the LPS are summarized as follows: 

□ Capture Raw ETM+ science data in real-time at 75-Mbps directly to disk 

□ Synchronize Major Frames and perform error detection and correction^ 

□ Identify subinter vals, scenes and scan line offse ts 

□ Generate bands (1-6L, 6H-8), metadata, and browse files 

□ Perform Cloud Cover Assessment and Scene Quality scoring 

□ Interface with the ECS for LOR subinterval data transfer and ingest 

□ Produce real-time moving window display during processing 

□ Facilitate systems operations, status monitoring, configuration, activity scheduling, and management 

3.1 Hardware Architecture 

The operational LPS hardware consists of five Silicon Graphics, Inc. (SGI) Challenge XL computers, two Indy workstations, 
and two X-terminals. Each SGI Challenge XL is configured with eight CPUs, 512 MB of RAM, a VME interface module, a 
custom built VME-9U-VSIO^ high-speed serial interface board, approximately 178 GB of Ciprico RAID external disk drives, 
a DLT 4000 library, and a DLT 7000 tape drive. Four of the five SGI systems are required for nominal operations including 
raw ETM+ data recording, science data processing, and data transfer. The fifth system is a "warm-spare" supporting 
activities such as training, software maintenance, and testing. The Indy workstations and X-terminals are used as operational 
terminals supporting the LPS graphical user interfaces and the moving window displays. The custom built VME-9U-VSIO 
high-speed serial interface board, inserted into each SGI XL VME backplane, accepts as input the 75-Mbps serial differential 


“ Note: A valuable tool used to validate the raw ETM+ stream was the DataScope by Jackson and Tull, Seabrook, Md. 
^ Note: The VME-9U-VSIO card was built by General Standards Corporation, Huntsville, Alabama 


Emitter Coupled Logic (ECL) clock and data signals from the LGS. A SGI developed custom device driver interfaces the 
VME-9U-VSIO card to the SGI IRIX operating system. 

3.2 Software Architecture 

The software component of the LPS consists of an Oracle database and seven software subsystems. The seven subsystems 
are Raw Data Capture, Raw Data Processing, Major Frame Processing, Payload Correction Data Processing, Image Data 
Processing, Data Transfer, and Process Management and Control. All inter process communication takes place via shared 
memory or an Oracle database. The user interface is based on Oracle Forms and Menus. 

□ The Raw Data Capture Subsystem is responsible for capturing raw ETM+ wideband data in real-time to a file for 
subsequent processing. It generates a raw data file of unsynchronized Consultative Committee for Space Data 
Systems (CCSDS) major frames and an ASCII metadata file with contact period information in addition to the 
generation of a data record in the database. The raw data capture process interfaces with a custom device driver 
developed by SGI that performs direct memory accessing for data ingest and transfer. 

□ The Raw Data Processing Subsystem accepts the raw data file from the raw data capture subsystem, performs frame 
synchronization, annotates the channel access data units, performs error detection and correction, generates data 
quality and accounting statistics, and transfers the data to the major frame processing subsystem 

□ The Major Frame Processing Subsystem is responsible for identifying a major frame, filling missing frames, 
determining major frame time and the subinterval, deriving band data offsets, generating the calibration and mirror 
scan correction data files, extracting the virtual channel data unit status information, extracting payload correction 
bytes, and generating the quality and accounting data 

□ The Payload Correction Data Processing Subsystem is responsible for generating the payload correction file, 
identifying each scene in accordance with the Worldwide Reference System (WRS) on a subinterval basis, and 
determining the Sun azimuth / elevation, horizontal display shift, and calibration door activity for each scene. 

□ The Image Data Processing Subsystem is responsible for generating the nine Landsat 7 band files (B1-6L, 6H-8), a 
browse image, the moving window display for operations, a cloud coverage assessment score, and accounting 
information that is inserted into the metadata files. 

□ The Data Transfer Subsystem interfaces with the ECS. It notifies the ECS of data availability, receives the data 
transfer acknowledgements and deletes data after successful transfer 

□ The Process Management and Control Subsystem is responsible for all system-level control arfd monitoring of the 
LPS devices and processes and provides the interface between the LPS and the operator. It facilitates updating the 
LPS processing parameters and thresholds, contact schedules, and automatic or manual execution of system tasks 
such as raw data capture and Level Zero processing. 

3.3 Concept of Operations 

An operator configures four of the five LPS computers with a daily contact schedule received from the MOC via the shared 
Oracle database. From this schedule, operational computers automatically perform data capture, processing, and tr^sfer 
functions. The operator views gross image quality and the progress of the data Level Zero processing with the dynamically 
updated moving window display. 

The nominal data processing flow is as follows: during either real-time contact with the satellite or tape playback the LPS 
SGI systems record the raw bit stream directly to RAID disk. After the raw data capture event is complete, the LPS initiates 
Level Zero and Image Processing for LOR subinterval generation. Upon completion of the processing a data-availability- 
notice is sent to the ECS for data transfer scheduling. When the ECS is ready, it ingests the LOR data subintervals and 
acknowledges data transfer status to the LPS. In support of potential reprocessing requests each operational LPS SGI makes 
a short-term copy of the raw data with the DLT 4000 tape library. 


3.4 Management Aspects 


The LPS project began in June of 1994. It was designed, developed, tested and deployed over a period of approximately 5 
years including the last 18 months of maintenance. The LPS software is sized at approximately 100,000 lines of Deliverable 
Source Instructions (DSl) including C code, header files, and Oracle Forms and Menus. The most significant issue that 
permeated every facet of the LPS development was the lack of good prelaunch data from the Landsat 7 instrument. As the 
result of no prelaunch data non-trivial format discrepancies, e.g., extra 0 byte, resulted in significant software changes at a 
time when development staff profiles were shrinking to maintenance levels. Additionally, some of the LPS capabilities, e.g., 
cloud cover assessment or WRS accuracy, could not be validated until the spacecraft was on nominal orbit. This necessitated 
the retention of highly skilled software maintenance personnel, which was incongruent to the staffing budgets. The success of 
the LPS project is attributed to the patience and thorough testing by operations personnel, industrious and responsiveness of 
the development personnel, and highly dedicated engineering and development team members. 

4. OVERVIEW - IMAGE ASSESSMENT SYSTEM (IAS) 

The Image Assessment System (IAS) is responsible for image quality assessment and instrument calibration^. In assessing 
image quality, IAS monitors and calibrates the radiometric and geometric performance of the ETM+ sensors. The calibration 
parameters are stored in a Calibration Parameter File (CPF) that is subsequently bundled with the LOR products and 
distributed by the ECS. Up to 10 LOR images are retrieved from the ECS each day and processed by the IAS to Level 1 
radiometrically corrected (LIR) and Level 1 geometrically corrected (LIG) format. The IAS provides the tools needed by 
the Landsat Mission Management Office to analyze and manage instrument or data format anomalies. 

The major functions of the LAS are summarized as follows: 

□ Assess Landsat 7 image quality to ensure compliance with the radiometric and geometric requirements of the 
spacecraft and the ETM+ sensor 

□ Generate Level 1 products that are radiometrically and geometrically corrected 

□ Compensate for inoperable and saturated detectors, and image artifacts of striping, banding, coherent noise, memory 
effect, scan correlated shift, and dropped scan lines 

□ Produce geometric products that have been systematically corrected, precision corrected, and terrain corrected 

□ Produce ETM+ calibration information using Internal Calibrator (IC), Partial-Aperture Solar Calibrator (PASC), and 
Full-Aperture Solar Calibrator (FASC), and night scene data 

□ Generate and archive ETM+ trending data for long-term analysis 

□ Produce radiometric and geometric calibration updates to the CPF 

4.1 Hardware Architecture 

The operational IAS hardware consists of one SGI Origin 2000 computer and three SGI 02 workstations. The Origin 2000 
contains four RlOK CPUs, two GB of RAM, and has 288 GBs of Ciprico RAID disk drives attached. The 02 workstations 
are each equipped with a four GB disk drive, 512MB RAM, and monitors with a 1280x1024 resolution. The Origin 2000 
performs all of the processing, and the 02 workstations perform data and image analysis, anomaly investigation, and report 
generation activities. One of 02 workstations was designed to serve as a system console for the Origin 2000. 

4.2 Software Architecture 

The IAS software architecture incorporates custom C programming language and several commercial-off-the-shelf (COTS) 
products. The architecture was designed to minimize IAS software development time. The COTS products include the 
Oracle Database Management System, Environment for Visualizing Images ^NVI), and Interactive Data Language (IDL). 
All inter-process communication occurs via the Oracle database or shared memory. The operator user interface is provided 
through Oracle Fornis and the analyst user interface is provided through ENVI and IDL. Hierarchical Data Format (HDF) 
tools and Application Programming Interfaces (API) are used to read and write the LOR scenes received from the EDC. 
Adobe FrameMaker is used for producing IAS reports. The IAS software consists of the following subsystems: Process 
Control, Data Management, Radiometric Processing, Geometric Processing, and Evaluation and Analysis. 

□ The Process Control Subsystem provides capabilities for planning, monitoring, and controlling work orders. It 
permits the operator to set up work orders specifying the processing to be performed on the data, to prioritize 



the processing to be performed, and to monitor the status of the work order execution. Process Control perfonns 
automated work order execution by monitoring required and available system resources, initiating application 
processing, and monitoring application execution status. It controls the execution of radiometric and geometric 
processing by initiating and managing the Level 1 processing. 


a 


■ . .. □ 


The Data Management Subsystem ingests, archives, and distributes the IAS data. The data consist of image 
data, ancillary data, characterization and calibration results, and reports. Data Management transfers data during 
Ingest using Tile Transfer Protocol (FTP) functions, stores data in the appropriate internal data stores, updates 
data availability information, and manages disk resources. For data export. Data Management performs the 
appropriate data formatting and supports the FTP transmission of these data to external systems. The Data 
Management software also performs quality checking and LOR correction upon receipt of the data from the 
ECS. It prepares data at the start of image processing by correcting the data based on the payload correction 
data (PCD) and the mirror scan correction data (MSCD). 

the Radiometric Processing Subsystem perfoms'ra^ calibration and LIR processing. Radiometric 

calibration determines the calibratibh ih-fllght of each detector, that is, the conversion from digital number to 
absolute radiance. The primary purposrbf ffirtTT processing is to convert the brighmess of the image pixels 
to absolute radiance. This process uses various ground- and in-flight-determined calibrations, i.e.. Internal 
Calibrator, PASC, lASC, and hlghtlcenTsl’IVadrtlb the Radiometric Processing Subsystem characterizes 
the quality and various features of the data, .e.g.. Inoperable detectors, image artifacts etc. ■ 


□ The Geometric Processing Subsystem perfonns geometric calibration and LIG processing. This subsystem 
processes raw LOR or radiometrically corrected (LIR) imagery into geometrically corrected (LIG) imagery. 
Additionally, it characterizes the quality and various features of the data. The LIG image can be processed to 
three levels: systematic (LlGs), precision-corrected (LlGp), and precision/terrain-corrected (LlGt). 

The IAS Geometric Processing Subsystem was best described by Jim Storey, USGS: 'The overall 
purpose of the IAS geometric algorithms is to use Earth ellipsoid and terrain surface information in 
conjunction with spacecraft ephemeris and attitude data, and knowledge of the Enhanced Thematic 
Mapper Plus (ETM+) instrument and Landsat 7 satellite geometry to relate locations in ETM+ 

- image space (band, scan, detector, sample) to geodetic object space (latitude, longitude, and height). 

These algorithms are used for purposes of creating accurate Level I output products, characterizing 
the ETM+ absolute and relative geometric accuracy, and deriving improved estimates of geometric 
calibration parameters such as the sensor to spacecraft alignment. " 


□ The Evaluation and Analysis Subsystem provides the tools required by the analyst to evaluate and analyze the 
performance of the ETM+ instrument, perform anomaly investigations, and maintain the CPF. Included are 
capabilities for viewing system input, processing reports and results (data analysis, statistical analysis and 
trending), and report generation. Additionally, the Evaluation and Analysis tools permit an analyst to invoke 
specific Mpects of radiometric and geometric processing that perform characterization and calibration functions. 
The results of the analysis coupled with trending results are used to update the CPF. The Evaluation and 
Analysis capability also provides the analyst capabilities for editing system input and intermediate results and 
for submitting processing requests in support of “what if’ analysis. 


4.3 Concept of Operations 

The IAS analyst orders up to 10 LOR products per day from the EDC DAAC for analysis and creates a work order, i.e., 
procedure for processing a LOR image to a LI image, for each. The IAS software automatically ingests the LOR data and 
schedules it for processing according to the work order specifications. It assesses the LOR input data and processes it to UR 
and LIG for assessment as LI products. Image data are assessed with respect to their radiometric and geometric qualities 
both on an individual product basis and for long-term trend analyses. 

The primary product of the IAS is the CPF. As part of this process the IAS submits special imaging requests to the MOC for 
partial aperture solar calibrator (PASC), full aperture solar calibrator (FASC), or calibration lamp cycling. The IAS analysts 
use the individual assessments of the LORs and the trending data to determine the updates to the CPF and generate reports 
concerning Landsat 7 instrument performance. These reports, along with an updated CPF when applicable, are provided to 


the MOC and the ECS. Users, who order LOR products from the ECS, subsequently receive this CPF file as part of their 
order. 

4.4Management Aspects 

The IAS project began late in 1994 and was reorganized in June 1996 thereby compressing the development phase. In 
February 1998 Release 2.0 was delivered to operations for acceptance testing. Release 3.0 following in June 1998 and it 
consisted of 223,000 DSI, which included header files, C source code, Oracle Forms and Menus, IDL, Unix Scripts, and SQL 
scripts. Between June 1998 md July 1999 one software release and eight patches were generated due to new requirements, 
data format changes, and software errors adding 1500 DSI to the total. The most significant challenges for the IAS 
development were the compressed software development schedule coupled with extremely complex scientific algorithms to 
implement and the lack of LOR scenes to test with. The compressed schedule necessitated tailoring the software development 
cycle by performing box level testing in lieu of unit testing. The lack of LOR scene data necessitated the development of a 
complex software testing tool that subsets, extracts, and generates Landsat 7 like Level OR scenes from Landsat 5 data. The 
success of the IAS project is attributed to the following: frequent working group meetings (2-3 per week), maintaining strict 
CM procedures, box level testing in lieu of unit testing, peer software inspections, thorough on-site testing by operations 
personnel, and highly dedicated and motivated badgeless development teams. 

5. OVERVIEW LEVEL 1 PRODUCTION SYSTEMS 

The Level 1 production systems consist of two distinct systems: the Level I Product Generation System (LPGS) and the 
Level 1 Product Distribution System (LPDS). The LPGS is responsible for acquiring LOR data associated with the Level 1 
orders, performing the requested radiometric and geometric processing, and generating the product in the user-specified 
format. The LPDS is responsible for receiving the LI orders from the EDC Billing and Accounting system, prioritizing them 
for processing, staging the orders for the LPGS, submitting orders for the associated LOR data to the ECS, and generating and 
distributing the final LI products. 

5.1 Level 1 Product Generation System (LPGS) 

The LPGS is a batch-oriented system driven by scripts that generate Landsat 7 Level 1 Products to the specifications 
provided by the customers. It interfaces with the LPDS and ECS for user product generation requests and data acquisition 
respectively. It can produce Level 1 products for a WRS scene or a floating subinterval sized between one half scene and 
three scenes. The LPGS is sized to sustain a daily volume of 100 WRS scene equivalent Level I products. 

The major functions of the LPGS are summarized as follows: 

1. Receive user product generation requests from the LPDS, signal LPDS when to order LOR data, and acquire the 
corresponding LOR data from the ECS to process the request 

2. Generate 100 LI WRS scene equivalent products per day in any combination of LIR or LIG products. Format LIR 
products in HDF. Format LIG products in FAST-L7A, HDF, or GeoTIFF 

3. Compensate for inoperable and saturated detectors, and image artifacts of striping, banding, coherent noise, memory 
effect, scan correlated shift, and dropped scan lines 

4. Produce geometric products that have been systematically corrected with the map projection, (oblique mercator, 
universal transverse mercator, lambert conformal conic, transverse mercator, oblique mercator, polyconic, and polar 
stereographic), resampling method (nearest neighbor, cubic convolution, or modulation transfer function), and 
orientation (north up or nominal path) selected by the user 

5.1.1 Hardware Architecture 

The operational LPGS hardware consists of one SGI Origin 2000 computer and two SGI 02 workstations, and an X-terminal. 
The Origin 2000 contains 16 RlOK CPUs, 24 GB of RAM, and 1.1 TB of Ciprico RAID external disk drives. The X- 
terminal is used for the operator interface, and the 02 workstations are used for product analysis when necessary. The 02 
workstations are equipped with 4 GB disk drives, 512 MB RAM, and monitors with a 1280x1024 resolution. A second SGI 
Origin 2000 is used for software maintenance and serves as a *‘cold-spare” to the operational system. It has eight processors, 
12 GB of RAM, and 288 GB of external RAID disk drives. 



5.1.2 Software Architecture 


The LPGS software architecture incorporates custom C software, IAS software reuse, and several COTS products. The 
COTS products include the Oracle Database Management System, ENVI, and IDL. The LPGS reused IAS’ Radiometric 
Processing and Geometric Processing Subsystems. Portions of the IAS Process Control and Data Management functions were 
also reused. The LPGS is similar architecture is similar to that of IAS. However, there are significant differences that reflect 
the minimal operator intervention criteria of LPGS, as well as the minimal analytical staff. All inter-process communication 
occurs via the Oracle database, script parameters, or Oracle pipes. The LPGS software consists of the following subsystems: 
Process Control, Data Management, Radiometric Processing, Geometric Processing, Quality Assessment, and Anomaly 
Analysis. 

□ Process Control Subsystem manages LPGS production planning and processing. It automatically takes product 
generation requests, i.e., specifications for customers order, and initiates monitors, and controls processing. 
Additionally, Process Control provides processing status in response to operator requests and manages LPGS 
resources for availability. 

□ Data Management Subsystem maintains and provides access to LPGS data stores. It handles communication 
protocols with the LPDS and ECS. Data Management ingests and formats files for use by its subsystems, 
providing cursory quality checks where needed to ensure the completeness of received and prepared products. 
It also provides formatting and packaging of LI output and makes these data available to the LPDS. 

□ Radiometric Processing Subsystem is reused from the IAS. The LPGS does not utilize all of the radiometric 
algorithms because it does not collect calibration data, nor does LPGS process PASC or FASC data. LPGS 
Radiometric Processing provides the results of characterizations performed and the processing status for use by 
IAS and the Quality Assessment Subsystem. 

□ Geometric Processing Subsystem is reused from the IAS Geometric Processing Subsystem. It creates 
systematically corrected LIG imagery from LIR products. However, it does not perform terrain or precision 
correction, nor are calibration data produced. 

□ Quality Assessment Subsystem generates and assembles postproduction information about image artifacts and 
effects that were not corrected, and it produces a summary of the processed image quality. It performs 
automated quality assurance after radiometric and geometric corrections of images have been made. 
Additionally, the Quality Assessment software provides capability for visual inspection of images after 
radiometric correction, geometric correction, and final product formatting. 

□ Anomaly Analysis Subsystem analyzes LI images and associated post production information to resolve image 
production anomalies. Typically the focus is on LOR products that could not be successfully processed to LI or 
that failed visual inspection. Additionally, using baselined LOR products. Anomaly Analysis permits an analyst 
to perform benchmark tests to confirm the integrity and performance of the LPGS after any system upgrades. 

5.1.3 Concept of Operations 

The LPGS automatically produces LI images as user requests are received from the LPDS. When resources are available the 
LPGS notifies the LPDS to post orders for LOR products in electronic format from the ECS. The LPDS receives notification 
of data availability and subsequently informs the LPGS for LOR data acquisition. The acquired LOR product is processed to 
LI as specified by the user (e.g., Level IG, GeoTIFF, space oblique mercator map projection, resampling method, etc.). 
During processing selected trending statistics are recorded for subsequent retrieval and processing by the IAS. LPDS 
retrieves the LI product from the LPGS, copies it to appropriate media, validates it, and notifies the LPGS of completion for 
subsequent file deletion. The LPGS system may be configured for manual visual quality inspection if necessary. 

5.1.4 Management Aspects 

Development of the LPGS began in August 1996, and with delivery of Release 3.0, in January 1999, all requirements had 
been implemented. Several maintenance deliveries followed that reflected changes in LPGS requirements, identified errors, 



3 nd changes in data format. When LPGS became operational, it totaled close to 220,000 DSI which included header files, C 
code, Oracle Forms and Menus, IDL, UNIX scripts, and SQL scripts. Of the 220,000 DSI, almost 135,000 are attributable to 
re-use of IAS, including radiometric and geometric processing software. 

The significant challenges for the development of the LPGS were the reuse of the IAS Radiometric and Geometric software 
subsystems and the late change in external interface requirements from ECS to LPDS. LPGS maximized the reuse of the IAS 
software but the LPGS' schedule was significantly behind the IAS, which made it very difficult to influence key requirements 
decisions. Often times circumstances did not permit reuse improvement changes to the IAS hence translation software w^ 
generated that reformatted user supplied processing parameters so that the radiometric and geometric subsystems could use it. 
Additionally, the LPDS was added in lieu of ECS in an effort to reduce the schedule for Level 1 product media generation 
and distribution to the launch plus 90 days time frame. The LPGS-ECS interface was replaced with the LPGS-LPDS 
interface necessitating a significant amount of software rework. In retrospect the success of the project can be attributed to 
highly dedicated professionals. 

5.2 Level 1 Product Distribution System (LPDS) 

The Level 1 Product Distribution System (LPDS) is the "middle man" of all Level 1 processing. It interfaces with the EDC 
billing and accounting system for receipt of approved data orders, the ECS for LOR data ordering, and the LPGS for LI 
processing. It contains custom software for managing the data orders, utilizes the EDC-built Product Distribution System 
^j)S) for media generation, and has a stand along server / application called the DAAC LI Server for the handling of LI 
data searches, browse requests, and orders as the ECS only recognizes orders for LOR data. All Level 1 product generation is 
performed in an "on-demand" first-in-first-out basis, assuming resource availability. The LPDS was designed to sustain the 
production of the equivalent of 100 WRS scenes per day. 

The major functions of the LPDS are summarized as follows: 

□ Facilitate the generation of Level 1 products and distribute them electronically or in 8-mm or CD-ROM 

□ Interface with the EDC billing and accounting system for receipt of LI orders and status reporting 

□ Coordinate generation of Level 1 products with the LPGS 

□ Provide product status information to EDC User Services 

□ Bridge Landsat 7 Level 1 inventory searches and orders from the VO IMS Client to the LOR based ECS 

5.2.1 Hardware Architecture 

The operational LPDS hardware consists of one SGI Origin 2000 computer, an SGI Origin 200 computer, a SGI 02 
workstation, two Rimage CD-RW Super Producers, several X-terminals, two networked Genicom label printers, and a 
Tektronix Phaser 740 Color Laser Printer. The SGI Origin 2000 has four RlOK CPUs, 768 MB RAM, and externally 
connected 216 GB of Ciprico RAID disks, six 8mm tape drives, and eight CD-R readers. The Origin 200 has one RlOK 
CPU, 640 MB of RAM, and 288 GB of Ciprico RAID disks connected to it. The Rimage Super Producers consist of a 
Pentium II PC coupled to a CD picker with four CD-RWs. The Super Producers can load, bum, and label CDs automatically 
and can hold up to 50 CDs at a time. The Origin 2000 is the backbone of all processing and the Origin 200 server supports 
the electronic file distribution. The SGI 02 workstation serves as a system console for the Origin systems and the X- 
terminals support the operator interfaces. The Color Laser Printer is used to generate a full-color thumbnail image of the 
user's product on the CD-ROM cover. The system was designed to produce a maximum of 200 CD-ROMs, 50 8-mm tapes, 
or stages 50 electronic files for transfers per day such that any combination of the media counts does not exceed the 
equivalent of 100 WRS scene products per day. 

5.2.2 Software Architecture 

The LPDS software incorporates custom C programming language with embedded Oracle SQL and Forms and the existing 
EDC-built Product Distribution System (PDS). The LPDS subsystems are the Order Manager, Acquire Manager, Ingest 
Manager, Product Distribution System, the Operator Interface, and the DAAC LI Server. 

□ The Order Manager Subsystem captures, validates, and acknowledges LI orders sent from the billing and 
accounting system and queues the order information for retrieval by the LPGS 


□ The Acquire Manager Subsystem coordinates LOR data ordering between the LPGS and ECS such that ordering 
only occurs when the LPGS is ready for more data. Additionally, order identification keys from the ECS and the 
billing and accounting system are collated to ensure product traceability 

a The Ingest Manager Subsystem coordinates between the LPGS and the PDS for the production of the final product 

□ The Product Distribution System Subsystem copies the Level 1 products to 8-mm tape, or CD-ROM or stages the 
data for electronic transfer to the user 

□ The Operator Interface Subsystem uses Oracle Forms to provide a menu-driven graphical user interface facilitating 
the management of all subsystems including order monitoring, promoting, resetting, and canceling 

□ DAAC LI Server Subsystem hides the details of the handling of the Level I products from the LO based ECS by 
intercepting and converting Level 1 directory and inventory searches, browse requests, and product orders to LOR 
directory and inventory searches, browse requests, and product orders. It also handles the conversion of LOR 
inventory searches to LI directory and inventory search results and forwards the browse images. 

5 . 2.3 Concept of Operations 

Via the EOS VO IMS^ web interface a customer searches and browses the Landsat archives at the EDC DAAC and submits 
an order for a Level 1 product. The order is sent to the EDC User Services for validation and approval. Upon approval the 
order is forwarded to the LPDS for processing via the billing and accounting system. 

The LPDS autonomously waits for the EDC billing and accounting system to send orders. All orders received by the LPDS 
are stored in a Level 1 order queue and passed to the LPGS based on a first-in-first-out basis, subject to media quotas and 
operator override direction. The orders progress through several intermediate processing queues as the associated LOR data 
are ordered, acquired, processed by the LPGS, and processed by the PDS. The LPGS stages the Level 1 products on its disk 
drives until the PDS copies the files, generates the specified media, and verifies the media. The operator manually loads and 
unloads tapes and CD-ROMs as prompted by the PDS for product verification. Additionally the operator collates media, 
labels, and delivery letters. Finally, the billing and accounting system is notified of order completion. 

5 . 2.4 Management Aspects 

Development of the LPDS began January 1998 and a launch ready version was delivered in February 1999. Release 2.0B 
was delivered to operations July 1999, and the final release was delivered in August 1999. The LPDS was designed, 
developed, and tested over a period of 20 months. Approximately 35,000 DSI, not counting the PDS, were developed, 
adapted, or converted during that period. The most significant challenge for the development of-the LPDS was the 
compressed schedule coupled with a large number of external interfaces. The success of the project is attributed to strict 
requirements control, simplification of interfaces, close collaboration with knowledgeable EDC DAAC development 
personnel, utilization of Landsat cognizant personnel, early software prototyping of key functions, and reusing the EDC-built 
PDS. 


6. OVERVIEW - EDC DAAC ECS 

The EDC provides one of eight DAACs of the Earth Observing System* (EOS) Data and Information System (EOSDIS). 
EOSDIS is one of the major elements of the EOS Program, a principal element of the Earth Science Enterprise (ESE). ESE 
is a long-term NASA research mission designed to improve our understanding of the Earth's interrelated processes involving 
the atmosphere, oceans, land surfaces, and polar regions’. EOSDIS is an end-to-end data system that include flight 
operations, data acquisition, data capture, initial processing, and backup archival. Data are routed to the DAACs and further 
processed into products for archival and distribution. Landat 7 only uses the DAAC portion of the EOSDIS for archival and 
distribution of data. Specifically, the data are held in the EOSDIS Core System (ECS). 


* See following URL for ordering L7 data http://harp.gsfc.nasa.gov/~imswivw/pub/imswelcome/ 



The ECS performs the following functions for Landsat: long-term data archiving, scene subsetting, and WRS LOR scene 
distribution. It also supports data search, browse, and order capabilities and interfaces with the LPS, IAS, and Level 1 
processing systems. The EDC DAAC holds the data for the Landsat 7, MODIS/Aster, Terra (AM-1), and Aqua missions. 

The Landsat 7 related functions of the ECS are summarized as follows: 

□ Interface with the LPS for receipt and archive of LOR subintervals, browse images, and associated metadata 

□ Correlate Landsat Format 1 and Format 2 subintervals during data ingest process 

□ Interface with International Ground Stations for receipt of browse images and metadata 

□ Receive, archive, and distribute the Landsat 7 CPF from the IAS 

□ Report WRS scenes archived and associated cloud coverage score to GSFC MOC 
a Interface with the EDC Billing and Accounting system 

□ Support customer search, browse, and order capabilities through EOSDIS Data Gateway 

□ Perform scene subsetting on a WRS boundary 

□ Ingest a maximum ofl40 GB of data from the LPS and up to 110 distribute 110 LOR products per day to the Level I 
Processing systems or customers. 

The ECS interfaces with the LPS for ingest of the processed Landsat 7 LOR subintervals, associated metadata, and browse 
images. A subset of the metadata and the browse images are aggregated into a database for search, browse, and order 
support. The EDC DAAC copies the data from the various LPS systems after notification of data availability is received. 
During the ingest and archive process the ECS must correlate the format 1 and format 2 subintervals because the data are 
received from independent computers within the LPS. When a scene is ordered the subinterval is recalled from off-line 
storage and staged for scene subsetting and subsequent LOR product generation, i.e., 8-mm tape or electronic transfer. 

7. SUMMARY 

The Landsat 7 Science Data Processing System went operational several days after spacecraft launch on April 15, 1999. The 
EDC DAAC began selling Landsat LOR and LI products in the fall of 1999. In 24 hours the Science Data Processing System 
can capture and archive the equivalent of 275 WRS scenes and can distribute the equivalent of 100 WRS LOR or LI scenes. 
The success of the Science Data Processing System is attributed to the dedicated professionals who developed and operate 
the systems. The systems are currently in operation at the EROS Data Center under the auspices of the USGS, Department of 
the Interior®. The LPS*, IAS, and LPGS software are available as freeware via the following methods. The LPS and LPGS 
software is available upon request from the L7 Program Manager. Please direct your inquires to the L7 Program Manager, 
EROS Data Center, U.S.Geological Survey, 47914 252"‘* St. Sioux Falls, SD., 57198. The IAS software may be obtained on 
line via http://landsat7.usEs.gov/ under the Technical Information and IAS Software Registration popup menu items. A PC 
adapted version of the LPGS called LPGS-LITE is available via the Open Source Remote Sensing (OSRS) web server at 
http://www.remotesensing.org/ . The COTS products used by these systems may be licensed from the applicable vendors. 
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